%Script used for processing LISST data that has been collected using the CHSD Profiler.
%
%This program computes loops through the ascii files containing software
%processed LISST data, with file names listed below. This script loops
%through the functions listed above, and outputs a mat file containing the
%burst average LISST data for each station (called filename.mat). 
%
%
% User will need to update "Parameters Section and process data using LISST
% software to convert .DAT or .LOG to .asc. (If not already done). 
%
%  IMPORTNAT NOTE: This is an interactive script. You will be given a
%  figure of depth/pressure (yaxis) versus time (along the x). You will
%  manually select the start and end time of each burst to averaged by clicking on
%  the corresponding data point on the figures (via matlab function
%  ginput).This is called infunciton convertEXO2_profile2MAT_v2.m 
%
%   FIRST click on the start and end time of the profiler down cast
%   (Continuous lowering of the  profile from surface to bottom at
%   beginning).
%
%   THEN click on the start and end of EACH of the sections corresponding to 2-5 minute burst samples.
%   Could be multiple sections (multiple depths, i.e. Bottom/Middle/Top) or
%   one depth (bottom). 
%
%   THEN HIT enter.
%   
%(For examples see powerpoint on Odrive: /CHSD-data/INSTRUMENT SOFTWARE/YSI EXO2 Sonde/Matlab_Processing_from_KAF_Apr2017Processing_CHSD_profile_index_selection.pptx)
%
%
% To process data using LISST software to convert .DAT or .LOG to.asc
%
%   1. Be sure correct to note volume calibration constant (VCC) is used and other instrument calibration factors are correct.
%       The VCC may be changed (old way of processing) in accordance to the Pathlength reduction module (PRM) that is used. This will
%       be either none, 50%, 80% or 90%. More details a few lines down. Other calibration factors should not need
%       to be changed unless the instrument is damaged, sent back for fixing, and returned with updated parameters. 
%
%       To check: open software: from menu bar along the top choose DataFrames--calibration parameters-- and use drop down select menu to select instrument by serial number.
%
%       You will see the values the LISST software is using. 
%
%       To be sure these are correct or change: See LISST.ini and Instrumentdata.txt files. Found on computer with the software under Program Data/Sequoia/LISST100.
%
%       VCC values in InstrumentData.txt. (FILE MUST ALWAYS MUST USE THIS SAME NAME!!!) The value in the 4 column of this file is the VCC.  
%       You will notice other files that represent varations in this file adjusted backed on the PRM used and names accordingly (i.e.
%       InstrumentData_1185_80percent.txt is the InstrumentData.txt file,with the VCC for SN1185 set for using a 80% PRM module). If
%       you do compare (and do some math) you will notice the VCC in InstrumentData_1185_80percent.txt is the value in 
%       InstrumentData.txt reduced by 80% (i.e multiply by 20% or 0.2). 
%
%      You need to note whether this is being changed to represent the PRM or left assuming 100 or no PRM used. You will have to tell the code this using a
%      logical variable "correctVCC" in the parameters section. 
%       
%       NEVER EVER DELETE or WRITE over the file InstrumentData_100percent.txt!!! This is from the Sequioa
%
%       Other calibration parameters in LISST.ini (FILE MUST ALWAYS MUST ALWAYS USE THIS NAME!!!)
%
%
%   2. Process with Batch Processing
%       In software:from menu bar along the top choose File--Open Batch Processing. A gui will pop up and request the following inputs
%           via drop down bar or browse option:
%   
%           1. Insturment serial Number:
%           2. Background File (i.e zsc file associated with that cruise).
%           3. Raw Files Directory: Location of .log files
%           4. Output Files Directory: locaiton of processed data. Files that are put 
%              here and will include .asc and .psd formats. "_rs.asc"
%               was processed using a random shape matrix for particle size
%               inversion, while ".asc" was processed using the original
%               spherical inversion matrix based on mie theory.
%               Click Process! 
%
%Required Functions called be sure they are in your matlab path: 
%
%
%Calls the following functions or scripts (Be sure they are in your matlab path or in the folder you are running this in):
%       
%   1. LISST100Xasc2mat_v2.m: Converts .asc files to .mat files, calculates area concentration distirubiton, 
%      and calculates additional particle size parameters. Calls function datanumfromdata.m and compute_mean_v2.m. 
%      See function for more information. 
% 
%           Calls the following functions
%                   A. datenumfromdata.m: Function to compute date in a year from columns 39 and 40 from the .dat
%                   B. compute_mean_v2.m
%
%
%    2.getscat_kaf.m:Function to get the scattering, transmission and zscat from a binary. We use it to estimate scattering (b),
%                absorption (indirectly), and to double check beam attenuation through another approximation of tau.
%                REQUIRED FILES: ringarea.asc file, zscat.asc file and tt2mat.m
%                The OUTPUTS are:
%                     tau_2: the optical transmission
%                     c_2: beam attenuation [m^-1]
%                     b:Near forward scattering coefficient [m^-1]  
%                     a:Calculated absorption coefficiet [m^-1]
%
%                    
%    3. getLISST100X_AVGdata_fromprofiler_v2.m: Computes burst avearges. 
%
%    4.create_data_struct_csv_profiler_LISST100X.m: Compiles all samples
%    from 1 cruise and saves as a .csv file and matlab strucutre
%
% Based off GetLISST100Xdata_CTDpress.m Orginially written by G. Massey 
%
%KAF April 2017
%
%% Parameters
SN=1239;%Instrument Serial number
InvType=3; %Inversion Type. For Random Shape LISST100x Type C InvType=31, LISST100x Type B InvType=21. For Spherical:LISST100x Type C InvType=3,LISST100x Type C InvType=2
PRM=0; %Path Length Reduction Module used, either 0 (non), 50 (50%), 80 (80%), or 90 (90%)
LISSTdir=('/Users/kelsey/Desktop/Dissertation/Profiler Surveys/York River/Axial Surveys/YR160907/LISST/Sphere/');%pathname to location of raw data
Cruise='YR160907'; %name you want to give it, USE _rs to denote if random shape inversion was applied 
year=2016;% year of deployment
%taulim=0.01; %tranmission limit for "good" data. Most literature suggests 0.2-0.3, but I kept it low because we were very very turbid. 


% Data Output path
fout='/Users/kelsey/Desktop/Dissertation/Profiler Surveys/York River/Axial Surveys/YR160907/LISST/Sphere/Test New Code/'; %Location to put processed files
if ~exist(fout,'dir') 
    mkdir (fout)
end

%Files required for processing files
zscfile='/Users/kelsey/Desktop/Dissertation/Profiler Surveys/York River/Axial Surveys/YR160907/LISST/Random Shape/zsc160818_1239_80percent.asc'; %path and name of to background zscat file
ringarea='/Users/kelsey/Desktop/Dissertation/Profiler Surveys/York River/Axial Surveys/YR160907/LISST/Random Shape/Ringarea_1239.asc'; %path and name of ring area file. Find copy on O drive /CHSD-data/INSTRUMENT SOFTWARE/Sequoia/ringarea_SNXXXX.asc)

%Name of LISST files. 
LISSTname=cellstr(['L5720'
'L5721'
'L5722'
'L5723'
'L5724'
'L5725'
]);


%Enter Distance from the York River Mouth (or reference point of your choice). MUST have a distance corresponding to each data file, even if
%they were all taken at the same place. 

distance=[11
    18
    30
    39
    45
    60];

%Logical to denote correcting VCC for PRM (if it wasn't already done when processing with software (i.e. editing 4 column value in
%InstrumentData.txt). 
%True if column 4 was left for a VCC=100% and false if it was changed to correspond with PRM
correctVCC=false;

%Logical to denote processing for absorption and forward scattering. Can
%only be done if raw data files are available.  True if *.log files available and scatter can be created. 
iRAW=true; %flag for raw  (*.log files). 

%% Bin sizes (Code from Sequoia). These are based on instrument type and inversion matrix used which is defined by the input "InvType"
rho=200^(1/32);

if InvType==2
    bins(:,1) = 1.25*rho.^([0:31]); %lower limit for type B
    bins(:,2) = 1.25*rho.^([1:32]); % upper limit for type B
    bins(:,3) = sqrt(bins(:,1).*bins(:,2));%mid-point for type B
    dias32 = bins(:,3);
    upperBins = bins(:,2);
    size100x= bins(:,3);
elseif InvType==3
    bins(:,1) = 2.5*rho.^([0:31]); %lower limit for type C
    bins(:,2) = 2.5*rho.^([1:32]); % upper limit for type C
    bins(:,3) = sqrt(bins(:,1).*bins(:,2));%mid-point for type C
    dias32 = bins(:,3);
    upperBins = bins(:,2);
    size100x= bins(:,3);
    
elseif InvType==21
    dias32 = [1.0863095E+00  1.1800684E+00; 1.2819196E+00  1.3925615E+00; 1.5127528E+00  1.6433178E+00;...
        1.7851518E+00  1.9392274E+00; 2.1066013E+00  2.2884211E+00; 2.4859336E+00  2.7004934E+00;...
        2.9335718E+00  3.1867670E+00; 3.4618154E+00  3.7606031E+00; 4.0851790E+00  4.4377689E+00;...
        4.8207907E+00  5.2368710E+00; 5.6888629E+00  6.1798660E+00; 6.7132474E+00  7.2926647E+00;...
        7.9220913E+00  8.6058433E+00; 9.3486097E+00  1.0155484E+01; 1.1031999E+01  1.1984166E+01;...
        1.3018514E+01  1.4142136E+01; 1.5362737E+01  1.6688688E+01; 1.8129081E+01  1.9693793E+01;...
        2.1393555E+01  2.3240023E+01; 2.5245859E+01  2.7424818E+01; 2.9791841E+01  3.2363161E+01;...
        3.5156411E+01  3.8190744E+01; 4.1486970E+01  4.5067691E+01; 4.8957463E+01  5.3182959E+01;...
        5.7773156E+01  6.2759530E+01; 6.8176276E+01  7.4060540E+01; 8.0452671E+01  8.7396504E+01;...
        9.4939656E+01  1.0313385E+02; 1.1203529E+02  1.2170500E+02; 1.3220931E+02  1.4362023E+02;
        1.5601603E+02  1.6948170E+02; 1.8410959E+02  2.0000000E+02];%mid points (column 1) and upper bins (column 2) for type B, randomly shaped

    upperBins = dias32(:,2);
    size100x=dias32(:,1);
    
elseif InvType==31
    dias32 = [2.05970200000000,2.24514560000000;2.43230210000000,2.64942550000000;2.87230560000000,3.12650330000000;...
        3.39190560000000,3.68948780000000;4.00550140000000,4.35384800000000;4.73009660000000,5.13783860000000;...
        5.58577110000000,6.06300100000000;6.59623700000000,7.15475600000000;7.78949630000000,8.44310160000000;...
        9.19861620000000,9.96343760000000;10.8626460000000,11.7575380000000;12.8276990000000,13.8746990000000;...
        15.1482290000000,16.3730940000000;17.8885440000000,19.3213720000000;21.1245810000000,22.8005400000000;...
        24.9460180000000,26.9061980000000;29.4587530000000,31.7511540000000;34.7878410000000,37.4685340000000;...
        41.0809620000000,44.2154340000000;48.5125080000000,52.1772370000000;57.2884200000000,61.5727100000000;...
        67.6518960000000,72.6600100000000;79.8901240000000,85.7437830000000;94.3422470000000,101.183530000000;...
        111.408760000000,119.403490000000;131.562600000000,140.904290000000;155.362280000000,166.276700000000;...
        183.467320000000,196.217880000000;216.656550000000,231.550520000000;255.849720000000,273.245460000000;...
        302.132940000000,322.448340000000;356.788790000000,380.511100000000];%mid points (column 1) and upper bins (column 2) for type C, randomly shaped

    upperBins = dias32(:,2);
    size100x=dias32(:,1);

else
    error(['You must specify a number: 1-4, 21 or 31, NOT ',num2str(InvType),'.']);
end
clear bins


%% Define Laser reference (Instrument specific). Used in calculating forward
%scattering and absorption (getscat.f). Provided by Sequioa (LISST.ini). 

if SN==1185
    HK3Scale=0.0005160; %Laser Reference Scale,mW (0 offset) from From LISST.ini for SN1185 (Instrument specific from LISST.ini) 
elseif SN==1239
    HK3Scale=0.0008738;
elseif SN==1629
    HK3Scale=0.0007915;%Laser Reference Scale,mW (0 offset) from From LISST.ini for SN1629 (Instrument specific from LISST.ini) 
end

%% Process Samples
for ii=1:length(LISSTname)
    if InvType==31 || InvType==21
        [vd,vd_A,vars]=LISST100Xasc2mat_v2([char(LISSTname(ii)),'_rs'],PRM,year,correctVCC,InvType,fout); 
    else
        [vd,vd_A,vars]=LISST100Xasc2mat_v2(char(LISSTname(ii)),PRM,year,correctVCC,InvType,fout); 
    end
    
%Calculate near forward scattering and absportion (indirectly) using getscat_kaf.m. 
%MUST HAVE RAW DATA FILE (.log) to use. If no .log file be iRAW=false. 

%Set X=1 for LISST-100X data
X=1;
if iRAW
[~,tau_2,c_2,b,a,~] = getscat_kaf_1239([char(LISSTname(ii)) '.log'],zscfile,1,ringarea,HK3Scale,PRM);
%[data,tau_2,c_2,b,a,cscat] = getscat_kaf_1239(datafile,zscfile,X,ringareafile,HK3Scale,PRM)
%[tau_2,c_2,b,a] = getscat_kaf_1239([char(LISSTname(ii)) '.log'],fullfile(LISSTdir,'zsc160602_1239.asc'),1,fullfile(LISSTdir,'ringarea_1239.asc'));
Ldata=getLISST100X_burstAVGdata_fromprofiler_noCTD(LISSTname(ii),fout,vars,vd,vd_A,b,a,c_2,tau_2);  
else
tau_2=NaN(length(vars));
c_2=NaN(length(vars));
b=NaN(length(vars));
a=NaN(length(vars));
Ldata=getLISST100X_burstAVGdata_fromprofiler_noCTD(LISSTname(ii),fout,vars,vd,vd_A,b,a,c_2,tau_2);  
end

clear tau_2 c_2 b a vd vd_A vars
end

% LdataC=Ldata; %corrected for wrong calibration constants and software
% version. Only needed for data collected before October 2014. 
% LdataC(:,2:33)=Ldata(:,4:35)./0.85;
% LdataC(:,37:39)=Ldata(:,37:39)./0.85;
% LdataC(:,43:75)=Ldata(:,43:75)./0.85;

%% Compiles all burst average LISST data from one curise into one matrix where each row corresponds to a different sample/depth, and
% create and save compiled data into .csv file and as a  matlab strucutre. 

Ldata=create_data_struct_csv_profiler_LISST100X_v2(Cruise,fout,size100x,distance,LISSTname);
    
save(fullfile(fout,Cruise),'-struct','Ldata'); 

